2019.3

Product Notices

On this page:

Breaking Changes

BREAKING CHANGE for 2019.3.15 - 2019.3.16

$where in Workflow Engine API Calls

Due to a vulnerability with MongoDB, the $where key is no longer supported as part of the query object for several API calls. The $where key is specifically used in MongoDB to support a JavaScript expression or function to be used as part of a query. Any APIs that were vulnerable to the usage of $where will now return with an error if $where is included in the query. The following APIs are affected by this, with the relevant parameter noted for each.

API Parameter
POST /workflow_engine/getJobFromTaskQuery task_query
POST /workflow_engine/queryJobs query
POST /workflow_engine/jobs/find query
POST /workflow_engine/queryTasksBrief query
POST /workflow_engine/getTask query


BREAKING CHANGE for 2019.3.0-2019.3.7 -> 2019.3.8

The following are breaking changes for the 2019.3.8 release.

Migration of array.toString Tasks to objectToString

For workflows using the array.toString task that was migrated or saved in a released version of IAP between 2019.3.0 and 2019.3.7, the array.toString task has now been migrated to objectToString. Between IAP versions 2019.3.0 and 2019.3.7, the array.toString task was changed to include two additional parameters and a slightly different output. Starting with IAP version 2019.3.8, the functionality of array.toString was reverted to the functionality that was prior to IAP version 2019.3.0. In other words, regarding the array.toString task, there is no longer a breaking change from 2019.2 workflows to its 2019.3 equivalent; however, in making this change, any workflows that intentionally used the new output of array.toString will now be broken as a result of this change. Accordingly, the next section outlines how to find and fix this scenario.

Manual Steps for Migrated Workflows

For workflows where the array.toString to objectToString migration is breaking, a script is now provided to find workflows that contain a specific task (this task can be used to find any task as the need arises). Follow the steps below to find all workflows that contain objectToString and manually update any that are breaking.

  1. Download the following script:

    Note: This script is also found on an installed version of 2019.3.8 in:

    /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

  2. Place the script in the following location (on the migrated environment that includes the 2019.3.8 maintenance release):

    /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

  3. Run the script to find all workflows that contain the objectToString task.

    cd /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

node findWorkflowTasks.js objectToString

  4. If necessary, update any workflow tasks that are now breaking by replacing them with array.ToString.

  5. Update any tasks that reference the output of array.ToString that have not been updated already by setting the reference task dropdown to stringified.

  6. Update any template variables that reference the previous arrayString output of array.ToString to now reference stringified. Remember to check any and all references that can occur subsequent to the array.toString task.

BREAKING CHANGE for 2019.3.5 -> 2019.3.6

RabbitMQ Properties Moved from Service Config to Profile

Upon upgrade to the 2019.3.6 maintenance bundle, the RabbitMQ properties will be automatically moved from all service configs to the profile. Additionally, these settings will be changed in the database; therefore all systems that share this database connection will be affected also. Please note, if you have a disaster recovery (DR) system in your setup, the changes made to the database will take effect once the first system is updated and affect all systems using that database. This basically means that all systems will need to be upgraded at the same time.

Rollback Instructions

Since the RabbitMQ properties will be auto-migrated from the service config to the profile, if a rollback is necessary, those properties must be restored manually to each relevant service config prior to upgrade.

BREAKING CHANGE for 2019.3.1/2019.3.2 -> 2019.3.3

The following are breaking changes for the 2019.3.3 release.

Important Note

It is the policy of Itential not to cause any breaking changes within the maintenance releases of a major release version; however, at times we have to do so and we extend our sincere apologies when those times occur. It is also our sincere hope that this breaking change was caught early enough so that most will not be affected.

Please note, Itential is removing two features:

  1. Translations, and
  2. Multiple job variables on the "Outgoing" tab of automatic tasks.

If you had previously elected not to utilize either of those features, then the steps shown below do not apply, and there is no need for you to complete them. As a best practice, however, there is no harm in double-checking to be sure whether or not this breaking change impacts you in any way.

Removing the Translations Feature

For the 2019.3.0 IAP release, Itential deprecated all the global object methods (string, number, etc.) in Workflow Builder and replaced them with a 'translations' feature. Once that deprecation was made, Itential heard from many customers that the manual effort to replace all those methods would be a very large one with the toolset that is provided. With that level of effort in mind, and our commitment to provide customers with the best user experience possible, Itential has decided to un-deprecate those methods in the 2019.3.3 maintenance patch.

Once you update to 2019.3.3 or higher, the methods will no longer be deprecated or have to be replaced. Please note, however, the new translations feature will be removed, which means you cannot use translations in your new workflows on development servers. The global object methods will become available beginning with IAP 2020.1.

Manual Steps to Ensure Database Integrity

Automations (within the Automation Studio) no longer allow tasks that export more than one job variable or that attach translations to exported job variables within automations. Workflow Engine will not support running automations with tasks that have translations on your outgoing data, or export more than one job variable as outgoing data. In the 2019.3.3 maintenance release, Workflow Engine will not start if it finds an automation that meets these conditions.

These automations need to be corrected manually. Use the following process:

On a system that is running 2019.3.1 or 2019.3.2 (prior to updating):

  1. Download the following two scripts:

    Note: These scripts are also found on an installed version of 2019.3.3 in:

    /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

  2. Place the script files in the following location of a 2019.3.1 or 2019.3.2 system:

    /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

  3. Run the script to find automations that have incompatible tasks.

    cd /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

node compatibilityCheck20193Maintenance.js

  4. The output of the script will indicate which automations need to be corrected.

    IMPORTANT: The correction must occur on a system running 2019.3.1 or 2019.3.2.

  5. After all automations have been updated, run the script again to ensure they are compliant.

  6. Once compliant, proceed with the update to 2019.3.3.

Sample Output

$ cd /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

$ node compatibilityCheck20193Maintenance.js

incompatibleWorkflow contains incompatible tasks:

    getAdaptersForDevice (508a)
    Task Summary: exports more than one job variable
    Reason: Task exports more than one job variable, each of which may or may not have translations attached to it.


    locateDevices (e7c)
    Task Summary: exports a job variable with a translation
    Reason: Task exports a job variable with a translation attached to it.


    locateDevices (37ec)
    Task Summary: exports more than one  job variable with translations
    Reason: Task exports more than one job variable, each of which may or may not have translations attached to it.


    RunCommand (2aed)
    Task Summary: exports more than one job var with no translations
    Reason: Task exports more than one job variable, each of which may or may not have translations attached to it.

Note: For ease of reference, the script output provides the name of the automation, e.g., incompatibleWorkflow, and the taskname, e.g., getAdaptersForDevice and locateDevices.

This compatibility check script inspects all the automations in your IAP deployment and will deem an automation incompatible if it contains any tasks that:

  • Export more than one job variable (each of which may or may not have a translation).

  • Export a job variable with a translation.

Modify the Automation to Fix Incompatibility

To fix the first case, modify the automation so that the tasks exporting more than one job variable now:

  • Only export a single job variable without any translations.

  • Use the newVariable task to instantiate the number of job variables that the incompatible task created.

  • Use the appropriate tasks from Number, Object, String, and Array to perform data manipulation on created job variables (if necessary).

Modify the Task to Fix Incompatibility

To fix the second case, the task on an incompatible automation should be modified such that:

  • It still exports a single job variable via its outgoing data.

  • All translations are removed.

  • The appropriate tasks from Number, Object, String, and Array are used to modify the exported job variable the same way that the translation did.

As stated above, once all workflows have been cleared of any tasks that have multiple job variables defined or have a translation, the update to 2019.3.3 is cleared to proceed.

BREAKING CHANGE for 2019.3.2 and below -> 2019.3.3

Job Manager Database Enhancement

The following procedure is only necessary if upgrading from 2019.3.2 or lower to release version 2019.3.3 or higher. If you have already performed these steps in an update to 2019.3.3 or above, then this procedure is not needed. There is no harm, however, in double-checking to be sure whether or not these steps may apply.

As part of an ongoing effort to always provide the best performance within IAP, this maintenance release contains a migration script which will optimize the jobs data to allow faster queries against the collection. There are two possible methods to run this migration:

  • npm run normalizeGroups from the app-workflow_engine directory. The script will modify each job within the database, so depending on the number of jobs it may take 5 - 15 mins to complete. This is the slowest method because it is run from the IAP server and is only recommended for instances where the jobs collection is moderately sized (low 10Ks number of records).

  • Download the normalizeGroups.js script and run it on the MongoDB server via the command line. For example: mongo mongodb://HOSTS:PORT/DATABASE?replicaSet=rs_pronghorn --authenticationDatabase admin -u USER -p PASS < normalizeGroups.js. Tailor the MongoDB URI to your environment. This method is much faster and is recommended for large jobs collections.

These changes should not have any impact on your usage of Job/Task history.

Note: With this breaking change, it is very important that you ensure all indexes are setup correctly on Jobs and Tasks. This can be done by running the npm run index command from within app-workflow_engine. If you are missing any indexes, it may take 15-30 mins to complete the index creation on large databases. Lastly, always make sure you have a full MongoDB backup of your database before upgrading.

BREAKING CHANGES for 2019.2.* -> 2019.3

The following are breaking changes for the 2019.3 release.

app-workflow_engine

Removed the deprecated APIs for startJob, getCompletedJobs and getJobById. For replacement information, go to the Deprecations guide in Product Notices and see the Methods > Workflow Engine page.

app-workflow_builder

Removed the deprecated APIs for getWorkflowsFiltered and getWorkflowsList. For replacement information, go to the Deprecations guide in Product Notices and see the Methods > Workflow Builder page for IAP versions 2019.2 and below.

Updated Interface

The interface for both the workflow builder and list of workflows is now accessed through Automation Studio. There is no longer a shortcut icon on the IAP dashboard for this application. Go to Automation Studio to enjoy the exciting changes for automation creation!

app-form_builder

Updated Interface

The interface for both form builder and the list of forms is now accessed through Automation Studio. There is no longer a shortcut icon on the IAP dashboard for this application. Be sure to check out Automation Studio for form creation!

app-mop (Command Templates)

Updated Interface

The interface for both command templates and analytic templates is now accessed through Automation Studio. There is no longer a shortcut icon on the IAP dashboard for this application. Automation Studio is now your go-to for command and analytic template creation!

Removed Deprecated APIs

The methods following methods have been removed from app-mop (Command Templates) and are no longer available in the platform beginning in IAP 2019.3: executeShell, getBootFlash, getMOP, list, outputDiff, runCommand, setBoot (/copyImage), Wait.

app-template_builder

New Format of Jinja2 Templates Prior to Maintenance Release 2019.2.4

Although many will view this as an improvement, for those who have been using this relatively new feature, we would be remiss if we did not notify you of this change. In maintenance release 2019.2.4, the template provided in the Template Builder user interface no longer requires an opinionated reference to the 'data' variable. Before the change, you could supply any JSON, and then reference it in the template as "data.whatever." After the change, you can only supply a JSON object, and you no longer need to use the data prefix in the template. You can just reference the object properties identified in the JSON object by name directly. We feel this ease of use improvement warrants a change notice and aligns with the overall user experience within our product. We also hope to have caught this early enough to not have caused too much rework. If you have any templates built before the 2019.2.4 maintenance release, please update them as well as their corresponding JSON data object to match the new interface.

Updated Interface

The interface for viewing and building templates is now accessed through Automation Studio. There is no longer an icon on the IAP dashboard for this application. We're confident you'll enjoy the exciting changes in Automation Studio for template creation!

app-array

This application has been completely removed, and all tasks previously within this application have been moved inside of app-workflow_engine (which appears under the 'WorkFlowEngine' heading within the Workflow Builder Task list). The previous routes inside app-array will not exist, and no new routes have been created to replace them. Tasks will still appear under the Array tab.

app-json

This application has been completely removed, and all tasks previously within this application have been moved inside of app-workflow_engine (which appears under the 'WorkFlowEngine' heading within the Workflow Builder Task list). The previous routes inside app-json will not exist, and no new routes have been created to replace them. The JSON tab has also been removed. Stringify has been moved to the Object tab and renamed to toString. Parse has been moved to the String tab.

The Object.toString and Parse tasks will be automatically migrated so no manual intervention is needed, and the workflow will behave the same as it did in 2019.2.

app-number

This application has been completely removed, and all tasks previously within this application have been moved inside of app-workflow_engine (which appears under the 'WorkFlowEngine' heading within the Workflow Builder Task list). The previous routes inside app-number will not exist, and no new routes have been created to replace them. Tasks will still appear under the Number tab.

The tasks in app-number will be automatically migrated so no manual intervention is needed, and the workflow will behave the same as it did in 2019.2.

app-object

This application has been completely removed, and all tasks previously within this application have been moved inside of app-workflow_engine (which appears under the 'WorkFlowEngine' heading within the Workflow Builder Task list). The previous routes inside app-object will not exist, and no new routes have been created to replace them. Tasks will still appear under the Object tab.

The tasks in app-object will be automatically migrated so no manual intervention is needed, and the workflow will behave the same as it did in 2019.2.

app-string

This application has been completely removed, and all tasks previously within this application have been moved inside of app-workflow_engine (which appears under the 'WorkFlowEngine' heading within the Workflow Builder Task list). The previous routes inside app-string will not exist, and no new routes have been created to replace them. Tasks will still appear under the String tab.

app-tools

This application has been completely removed, and all tasks previously within this application have been moved inside of app-workflow_engine (which appears under the 'WorkFlowEngine' heading within the Workflow Builder Task list). The previous routes inside app-tools will not exist, and no new routes have been created to replace them. The one exception to this is diffToHTML, which has changed its route from /tools/diffToHTML to /workflow_engine/diffToHTML and has been deprecated. Tasks will still appear under the Tools tab.

The tasks in app-tools will automatically migrate so no manual intervention is needed, and the workflow will behave the same as it did in 2019.2.

app-configuration_manager

Changed getDevicesFiltered to now require the 'limit' property. Previously if the 'limit' property was not provided or if 'limit=0' then the result would yield all devices. If there were a high number of devices, then this would cause timeouts or exception errors. For system integrity, we have removed the capability to list all.

Tags Application

Updated Interfaces

Both createTag and createTags were updated to handle additional cases not previously covered. Both functions now return an error object for negative cases.

Error Object Returned

{
   code,
   message: {
     error,
     [dataName]: data,
   },
 }
Error Object Part Description
code The error code when an error is processed (presently set to 422 for all errors).
error A string containing the reason for the error.
[dataName] The particular data element related to the error.


Rules for createTag
  • A tag name must have four (4) characters or more, or an error object is returned.
  • If the tag object contains an ID, the ID is stripped away and a new one is created if all previous and remaining tag creation rules pass.
  • A tag with the same name will cause an error object to be returned.
Rules for createTags
  • The interface now checks for the object type to be an array of tag objects or it will return an error object.
  • All tag names must be four (4) characters or more, or an error object is returned.
  • If a tag object contains an ID, the ID is stripped away and a new one is created if all previous and remaining tag creation rules pass.
  • The submitted tags array is examined for duplicates. Only the non-duplicates are added after passing the previous rules.